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ABSTRACT 


This study examines the evidence for the effectiveness of active 
learning. It defines the common forms of active learning most 
relevant for engineering faculty and critically examines the core 
element of each method. It is found that there is broad but 
uneven support for the core elements of active, collaborative, 
cooperative and problem-based learning. 
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I. INTRODUCTION 


Active learning has received considerable attention over the 
past several years. Often presented or perceived as a radical change 
from traditional instruction, the topic frequently polarizes faculty. 
Active learning has attracted strong advocates among faculty look- 
ing for alternatives to traditional teaching methods, while skeptical 
faculty regard active learning as another in a long line of educa- 
tional fads. 

For many faculty there remain questions about what active 
learning is and how it differs from traditional engineering educa- 
tion, since this is already “active” through homework assignments 
and laboratories. Adding to the confusion, engineering faculty do 
not always understand how the common forms of active learning 
difter from each other and most engineering faculty are not inclined 
to comb the educational literature for answers. 

This study addresses each of these issues. First, it defines active 
learning and distinguishes the different types of active learning 
most frequently discussed in the engineering literature. A core ele- 
ment is identified for each of these separate methods in order to dif- 
ferentiate between them, as well as to aid in the subsequent analysis 
of their effectiveness. Second, the study provides an overview of rel- 
evant cautions for the reader trying to draw quick conclusions on 
the effectiveness of active learning from the educational literature. 
Finally, it assists engineering faculty by summarizing some of the 
most relevant literature in the field of active learning. 


II. DEFINITIONS 
It is not possible to provide universally accepted definitions for 
all of the vocabulary of active learning since different authors in the 


field have interpreted some terms differently. However, it is possi- 
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ble to provide some generally accepted definitions and to highlight 
distinctions in how common terms are used. 

Active learning is generally defined as any instructional method 
that engages students in the learning process. In short, active learning 
requires students to do meaningful learning activities and think about 
what they are doing [1]. While this definition could include tradi- 
tional activities such as homework, in practice active learning refers to 
activities that are introduced into the classroom. The core elements of 
active learning are student activity and engagement in the learning 
process. Active learning is often contrasted to the traditional lecture 
where students passively receive information from the instructor. 

Collaborative learning can refer to any instructional method in 
which students work together in small groups toward a common goal 
[2]. As such, collaborative learning can be viewed as encompassing all 
group-based instructional methods, including cooperative learning 
[3-7]. In contrast, some authors distinguish between collaborative 
and cooperative learning as having distinct historical developments 
and different philosophical roots [8-10]. In either interpretation, the 
core element of collaborative learning is the emphasis on student in- 
teractions rather than on learning as a solitary activity. 

Cooperative learning can be defined as a structured form of group 
work where students pursue common goals while being assessed in- 
dividually [3, 11]. The most common model of cooperative learn- 
ing found in the engineering literature is that of Johnson, Johnson 
and Smith [12, 13]. This model incorporates five specific tenets, 
which are individual accountability, mutual interdependence, face- 
to-face promotive interaction, appropriate practice of interpersonal 
skills, and regular self-assessment of team functioning. While dif- 
ferent cooperative learning models exist [14, 15], the core element 
held in common is a focus on cooperative incentives rather than 
competition to promote learning. 

Problem-based learning (PBL) is an instructional method where rele- 
vant problems are introduced at the beginning of the instruction cycle 
and used to provide the context and motivation for the learning that 
follows. It is always active and usually (but not necessarily) collaborative 
or cooperative using the above definitions. PBL typically involves sig- 
nificant amounts of self-directed learning on the part of the students. 


TIT. COMMON PROBLEMS INTERPRETING THE 
LITERATURE ON ACTIVE LEARNING 


Before examining the literature to analyze the effectiveness of 
each approach, it is worth highlighting common problems that en- 
gineering faculty should appreciate before attempting to draw con- 
clusions from the literature. 


A. Problems Defining What Is Being Studied 
Confusion can result from reading the literature on the effec- 
tiveness of any instructional method unless the reader and author 
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take care to specify precisely what is being examined. For example, 
there are many different approaches that go under the name of 
problem-based learning [16]. These distinct approaches to PBL 
can have as many differences as they have elements in common, 
making interpretation of the literature difficult. In PBL, for exam- 
ple, students typically work in small teams to solve problems in a 
self-directed fashion. Looking at a number of meta-analyses [17], 
Norman and Schmidt [18] point out that having students work in 
small teams has a positive effect on academic achievement while 
self-directed learning has a slight negative effect on academic 
achievement. If PBL includes both of these elements and one asks if 
PBL works for promoting academic achievement, the answer seems 
to be that parts of it do and parts of it do not. Since different appli- 
cations of PBL will emphasize different components, the literature 
results on the overall effectiveness of PBL are bound to be confus- 
ing unless one takes care to specify what is being examined. This is 
even truer of the more broadly defined approaches of active or col- 
laborative learning, which encompass very distinct practices. 

Note that this point sheds a different light on some of the avail- 
able meta-analyses that are naturally attractive to a reader hoping 
for a quick overview of the field. In looking for a general sense of 
whether an approach like problem-based learning works, nothing 
seems as attractive as a meta-analysis that brings together the results 
of several studies and quantitatively examines the impact of the ap- 
proach. While this has value, there are pitfalls. Aggregating the re- 
sults of several studies on the effectiveness of PBL can be mislead- 
ing if the forms of PBL vary significantly in each of the individual 
studies included in the meta-analysis. 

To minimize this problem, the analysis presented in Section IV 
of this paper focuses on the specific core elements of a given instruc- 
tional method. For example, as discussed in Section II, the core ele- 
ment of collaborative learning is working in groups rather than 
working individually. Similarly, the core element of cooperative 
learning is cooperation rather than competition. These distinctions 
can be examined without ambiguity. Furthermore, focusing on the 
core element of active learning methods allows a broad field to be 
treated concisely. 


B. Problems Measuring “What Works” 

Just as every instructional method consists of more than one ele- 
ment, it also affects more than one learning outcome [18]. When 
asking whether active learning “works,” the broad range of out- 
comes should be considered such as measures of factual knowledge, 
relevant skills and student attitudes, and pragmatic items as student 
retention in academic programs. However, solid data on how an in- 
structional method impacts all of these learning outcomes is often 
not available, making comprehensive assessment difficult. In addi- 
tion, where data on multiple learning outcomes exists it can include 
mixed results. For example, some studies on problem-based learn- 
ing with medical students [19, 20] suggest that clinical performance 
is slightly enhanced while performance on standardized exams de- 
clines slightly. In cases like this, whether an approach works is a 
matter of interpretation and both proponents and detractors can 
comfortably hold different views. 

Another significant problem with assessment is that many rele- 
vant learning outcomes are simply difficult to measure. This is par- 
ticularly true for some of the higher level learning outcomes that are 
targeted by active learning methods. For example, PBL might nat- 
urally attract instructors interested in developing their students’ 
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ability to solve open-ended problems or engage in life-long learn- 
ing, since PBL typically provides practice in both skills. However, 
problem solving and life-long learning are difficult to measure. As a 
result, data are less frequently available for these outcomes than for 
standard measures of academic achievement such as test scores. 
This makes it difficult to know whether the potential of PBL to 
promote these outcomes is achieved in practice. 

Even when data on higher-level outcomes are available, it is easy 
to misinterpret reported results. Consider a study by Qin et al. [21] 
that reports that cooperation promotes higher quality individual 
problem solving than does competition. The result stems from the 
finding that individuals in cooperative groups produced better solu- 
tions to problems than individuals working in competitive environ- 
ments. While the finding might provide strong support for cooper- 
ative learning, it is important to understand what the study does zor 
specifically demonstrate. It does not necessarily follow from these 
results that students in cooperative environments developed 
stronger, more permanent and more transferable problem solving 
skills. Faculty citing the reference to prove that cooperative learning 
results in individuals becoming generically better problem solvers 
would be over-interpreting the results. 

A separate problem determining what works is deciding when 
an improvement is significant. Proponents of active learning some- 
times cite improvements without mentioning that the magnitude of 
the improvement is small [22]. This is particularly misleading when 
extra effort or resources are required to produce an improvement. 
Quantifying the impact of an intervention is often done using effect 
sizes, which are defined to be the difference in the means of a sub- 
ject and control population divided by the pooled standard devia- 
tion of the populations. An improvement with an effect size of 1.0 
would mean that the test population outperformed the control 
group by one standard deviation. Albanese [23] cites the benefits of 
using effect sizes and points out that Cohen [24] arbitrarily labeled 
effect sizes of 0.2, 0.5 and 0.8 as small, medium and large, respec- 
tively. Colliver [22] used this fact and other arguments to suggest 
that effect sizes should be at least 0.8 before they be considered sig- 
nificant. However, this suggestion would discount almost every 
available finding since effect sizes of 0.8 are rare for any intervention 
and require truly impressive gains [23]. The effect sizes of 0.5 or 
higher reported in Section IV of this paper are higher than those 
found for most instructional interventions. Indeed, several decades 
of research indicated that standard measures of academic achieve- 
ment were not particularly sensitive to any change in instructional 
approach [25]. Therefore, reported improvements in academic 
achievement should not be dismissed lightly. 

Note that while effect sizes are a common measure of the mag- 
nitude of an improvement, absolute rather than relative values are 
sometimes more telling. There can be an important difference be- 
tween results that are statistically significant and those that are sig- 
nificant in absolute terms. For this reason, it is often best to find 
both statistical and absolute measures of the magnitude of a report- 
ed improvement before deciding whether it is significant. 

As a final cautionary note for interpreting reported results, some 
readers dismiss reported improvements from nontraditional in- 
structional methods because they attribute them to the Hawthorne 
effect whereby the subjects knowingly react positively to any novel 
intervention regardless of its merit. The Hawthorne effect is gener- 
ally discredited, although it retains a strong hold on the popular 
imagination [26]. 
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C. Summary 

There are pitfalls for engineering faculty hoping to pick up an ar- 
ticle or two to see if active learning works. In particular, readers 
must clarify what is being studied and how the authors measure and 
interpret what “works.” The former is complicated by the wide 
range of methods that fall under the name of active learning, but 
can be simplified by focusing on core elements of common active 
learning methods. Assessing “what works” requires looking at a 
broad range of learning outcomes, interpreting data carefully, quan- 
tifying the magnitude of any reported improvement and having 
some idea of what constitutes a “significant” improvement. This last 
will always be a matter of interpretation, although it is helpful to 
look at both statistical measures such as effect sizes and absolute val- 
ues for reported learning gains. 

No matter how data is presented, faculty adopting instructional 
practices with the expectation of seeing results similar to those re- 
ported in the literature should be aware of the practical limitations 
of educational studies. Educational studies tell us what worked, on 
average, for the populations examined and learning theories suggest 
why this might be so. However, claiming that faculty who adopt a 
specific method will see similar results in their own classrooms is 
simply not possible. Even if faculty master the new instructional 
method, they can not control all other variables that affect learning. 
The value of the results presented in Section IV of the paper is that 
they provide information to help teachers “go with the odds.” The 
more extensive the data supporting an intervention, the more a 
teacher’s students resemble the test population and the bigger the 
reported gains, the better the odds are that the method will work for 
a given instructor. 

Notwithstanding all of these problems, engineering faculty 
should be strongly encouraged to look at the literature on active 
learning. Some of the evidence for active learning is compelling and 
should stimulate faculty to think about teaching and learning in 
nontraditional ways. 


TV. THE EVIDENCE FOR ACTIVE LEARNING 


Bonwell and Eison [1] summarize the literature on active learn- 
ing and conclude that it leads to better student attitudes and im- 
provements in students’ thinking and writing. They also cite evi- 
dence from McKeachie that discussion, one form of active learning, 
surpasses traditional lectures for retention of material, motivating 
students for further study and developing thinking skills. Felder 
et al. [27] include active learning on their recommendations for 
teaching methods that work, noting among other things that active 
learning is one of Chickering and Gamson’s “Seven Principles for 
Good Practice” [28]. 

However, not all of this support for active learning is compelling. 
McKeachie himself admits that the measured improvements of dis- 
cussion over lecture are small [29]. In addition, Chickering and 
Gamson do not provide hard evidence to support active learning as 
one of their principles. Even studies addressing the research base for 
Chickering and Gamson’s principles come across as thin with re- 
spect to empirical support for active learning. For example, Scorcelli 
[30], in a study aimed at presenting the research base for Chicker- 
ing and Gamson’s seven principles, states that, “We simply do not 
have much data confirming beneficial effects of other (not coopera- 
tive or social) kinds of active learning.” 
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Despite this, the empirical support for active learning is exten- 
sive. However, the variety of instructional methods labeled as active 
learning muddles the issue. Given differences in the approaches la- 
beled as active learning, it is not always clear what is being promoted 
by broad claims supporting the adoption of active learning. Perhaps 
it is best, as some proponents claim, to think of active learning as an 
approach rather than a method [31] and to recognize that different 
methods are best assessed separately. 

This assessment is done in the following sections, which look at 
the empirical support for active, collaborative, cooperative and prob- 
lem-based learning. As previously discussed, the critical elements of 
each approach are singled out rather than examining the effective- 
ness of every possible implementation scheme for each of these dis- 
tinct methods. The benefits of this general approach are twofold. 
First, it allows the reader to examine questions that are both funda- 
mental and pragmatic, such as whether introducing activity into the 
lecture or putting students into groups, is effective. Second, focusing 
on the core element eliminates the need to examine the effectiveness 
of every instructional technique that falls under a given broad catego- 
ry, which would be impractical within the scope of a single paper. 
Readers looking for literature on a number of specific active learning 
methods are referred to additional references [1, 6, 32]. 


A. Active Learning 

We have defined the core elements of active learning to be intro- 
ducing activities into the traditional lecture and promoting student 
engagement. Both elements are examined below, with an emphasis 
on empirical support for their effectiveness. 


1) Introducing student activity into the traditional lecture: On 
the simplest level, active learning is introducing student activity into 
the traditional lecture. One example of this is for the lecturer to 
pause periodically and have students clarify their notes with a part- 
ner. This can be done two or three times during an hour-long class. 
Because this pause procedure is so simple, it provides a baseline to 
study whether short, informal student activities can improve the ef- 
fectiveness of lectures. 

Ruhl et al. [33] show some significant results of adopting this 
pause procedure. In a study involving 72 students over two courses 
in each of two semesters, the researchers examined the effect of in- 
terrupting a 45-minute lecture three times with two-minute breaks 
during which students worked in pairs to clarify their notes. In par- 
allel with this approach, they taught a separate group using a 
straight lecture and then tested short and long-term retention of 
lecture material. Short-term retention was assessed by a free-recall 
exercise where students wrote down everything they could remem- 
ber in three minutes after each lecture and results were scored by the 
number of correct facts recorded. Short-term recall with the pause 
procedure averaged 108 correct facts compared to 80 correct facts 
recalled in classes with straight lecture. Long-term retention was as- 
sessed with a 65 question multiple-choice exam given one and a half 
weeks after the last of five lectures used in the study. Test scores 
were 89.4 with the pause procedure compared to 80.9 without 
pause for one class, and 80.4 with the pause procedure compared to 
72.6 with no pause in the other class. Further support for the effec- 
tiveness of pauses during the lecture is provided by Di Vesta [34]. 

Many proponents of active learning suggest that the effectiveness 
of this approach has to do with student attention span during lecture. 
Wankat [35] cites numerous studies that suggest that student 
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Average College and University Results 
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Figure 1. Active-engagement vs. traditional instruction for im- 
proving students’ conceptual understanding of basic physics concepts 
(taken from Laws et al., 1999) 


attention span during lecture is roughly fifteen minutes. After that, 
Hartley and Davies [36] found that the number of students paying 
attention begins to drop dramatically with a resulting loss in reten- 
tion of lecture material. The same authors found that immediately 
after the lecture students remembered 70 percent of information 
presented in first ten minutes of the lecture and 20 percent of infor- 
mation presented in last ten minutes. Breaking up the lecture might 
work because students’ minds start to wander and activities provide 
the opportunity to start fresh again, keeping students engaged. 


2) Promoting Student Engagement: Simply introducing activ- 
ity into the classroom fails to capture an important component of 
active learning. The type of activity, for example, influences how 
much classroom material is retained [34]. In “Understanding by 
Design” [37], the authors emphasize that good activities develop 
deep understanding of the important ideas to be learned. To do 
this, the activities must be designed around important learning out- 
comes and promote thoughtful engagement on the part of the stu- 
dent. The activity used by Ruhl, for example, encourages students 
to think about what they are learning. Adopting instructional prac- 
tices that engage students in the learning process is the defining fea- 
ture of active learning. 

The importance of student engagement is widely accepted and 
there is considerable evidence to support the effectiveness of student 
engagement on a broad range of learning outcomes. Astin [38] 


reports that student involvement is one of the most important pre- 
dictors of success in college. Hake [39] examined pre- and post-test 
data for over 6,000 students in introductory physics courses and 
found significantly improved performance for students in classes 
with substantial use of interactive-engagement methods. Test 
scores measuring conceptual understanding were roughly twice as 
high in classes promoting engagement than in traditional courses. 
Statistically, this was an improvement of two standard deviations 
above that of traditional courses. Other results supporting the effec- 
tiveness of active-engagement methods are reported by Redish et al. 
[40] and Laws et al. [41]. Redish et al. show that the improved 
learning gains are due to the nature of active engagement and not to 
extra time spent on a given topic. Figure 1, taken from Laws et al., 
shows that active engagement methods surpass traditional instruc- 
tion for improving conceptual understanding of basic physics con- 
cepts. The differences are quite significant. Taken together, the 
studies of Hake et al., Redish et al. and Laws et al. provide consider- 
able support for active engagement methods, particularly for ad- 
dressing students’ fundamental misconceptions. The importance of 
addressing student misconceptions has recently been recognized as 
an essential element of effective teaching [42]. 

In summary, considerable support exists for the core elements of 
active learning. Introducing activity into lectures can significantly 
improve recall of information while extensive evidence supports the 
benefits of student engagement. 


B. Collaborative Learning 

The central element of collaborative learning is collaborative vs. 
individual work and the analysis therefore focuses on how collabora- 
tion influences learning outcomes. The results of existing meta-stud- 
ies on this question are consistent. In a review of 90 years of research, 
Johnson, Johnson and Smith found that cooperation improved learn- 
ing outcomes relative to individual work across the board [12]. Simi- 
lar results were found in an updated study by the same authors [13] 
that looked at 168 studies between 1924 and 1997. Springer et al. 
[43] found similar results looking at 37 studies of students in science, 
mathematics, engineering and technology. Reported results for each 
of these studies are shown in Table 1, using effect sizes to show the 
impact of collaboration on a range of learning outcomes. 

What do these results mean in real terms instead of effect sizes, 
which are sometimes difficult to interpret? With respect to academic 
achievement, the lowest of the three studies cited would move a 


Reference 


Johnson, Johnson and Smith [12] 


Improved academic achievement 


Effect 
Size 


Learning Outcome 


Improved quality of interpersonal interactions 


Improved self-esteem 


Johnson, Johnson and Smith [13] 


Improved perceptions of greater social support 
Improved academic achievement 


Improved liking among students 


Improved self-esteem 


Improved perceptions of greater social support 


Springer et al. [43] 


Improved academic achievement 


Improved student attitudes 


Improved retention in academic programs 


Table 1. Collaborative vs. individualistic learning: Reported effect size of the improvement in different learning outcomes. 
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Reference Learning Outcome Effect 
Size 


Johnson, Johnson and Smith [12] | Improved academic achievement 
Improved interpersonal relationships 
Improved perceptions of greater social support 


Improved self-esteem 


Johnson, Johnson and Smith [13] | Improved academic achievement 


Improved liking among students 
Improved perceptions of greater social support 


Improved self-esteem 


Table 2. Collaborative vs. competitive learning: Reported effect size of the improvement in different learning outcomes. 


student from the 50% to the 70® percentile on an exam. In absolute 
terms, this change is consistent with raising a student’s grade from 
75 to 81, given classical assumptions about grade distributions.* 
With respect to retention, the results suggest that collaboration re- 
duces attrition in technical programs by 22 percent, a significant 
finding when technical programs are struggling to attract and retain 
students. Furthermore, some evidence suggests that collaboration is 
particularly effective for improving retention of traditionally under- 
represented groups [44, 45]. 

A related question of practical interest is whether the benefits of 
group work improve with frequency. Springer et al. looked specifical- 
ly at the effect of incorporating small, medium and large amounts of 
group work on achievement and found the positive effect sizes associ- 
ated with low, medium and high amount of time in groups to be 0.52, 
0.73 and 0.53, respectively. That is, the highest benefit was found for 
medium time in groups. In contrast, more time spent in groups did 
produce the highest effect on promoting positive student attitudes, 
with low, medium and high amount of time in groups having effect 
sizes of 0.37, 0.26, and 0.77, respectively. Springer et al. note that the 
attitudinal results were based on a relatively small number of studies. 

In summary, a number of meta-analyses support the premise 
that collaboration “works” for promoting a broad range of student 
learning outcomes. In particular, collaboration enhances academic 
achievement, student attitudes, and student retention. The magni- 
tude, consistency and relevance of these results strongly suggest that 
engineering faculty promote student collaboration in their courses. 


C. Cooperative Learning 

At its core, cooperative learning is based on the premise that co- 
operation is more effective than competition among students for 
producing positive learning outcomes. This is examined in Table 2. 

The reported results are consistently positive. Indeed, looking at 
high quality studies with good internal validity, the already large ef- 
fect size of 0.67 shown in Table 2 for academic achievement in- 
creases to 0.88. In real terms, this would increase a student’s exam 
score from 75 to 85 in the “classic” example cited previously, though 
of course this specific result is dependent on the assumed grade dis- 
tribution. As seen in Table 2, cooperation also promotes interper- 
sonal relationships, improves social support and fosters self-esteem. 

Another issue of interest to engineering faculty is that coopera- 
tive learning provides a natural environment in which to promote 


*Calculated using an effect size of 0.5, a mean of 75 and a normalized grade dis- 
tribution where the top 10 percent of students receive a 90 or higher (an A) and the 
bottom 10 percent receive a 60 or lower (an F). 
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effective teamwork and interpersonal skills. For engineering faculty, 
the need to develop these skills in their students is reflected by the 
ABET engineering criteria. Employers frequently identify team 
skills as a critical gap in the preparation of engineering students. 
Since practice is a precondition of learning any skill, it is difficult to 
argue that individual work in traditional classes does anything to 
develop team skills. 

Whether cooperative learning effectively develops interpersonal 
skills is another question. Part of the difficulty in answering that 
question stems from how one defines and measures team skills. 
Still, there is reason to think that cooperative learning is effective in 
this area. Johnson et al. [12, 13] recommend explicitly training stu- 
dents in the skills needed to be effective team members when using 
cooperative learning groups. It is reasonable to assume that the op- 
portunity to practice interpersonal skills coupled with explicit in- 
structions in these skills is more effective than traditional instruction 
that emphasizes individual learning and generally has no explicit in- 
struction in teamwork. There is also empirical evidence to support 
this conclusion. Johnson and Johnson report that social skills tend 
to increase more within cooperative rather than competitive or indi- 
vidual situations [46]. Terenzini et al. [47] show that students re- 
port increased team skills as a result of cooperative learning. In addi- 
tion, Panitz [48] cites a number of benefits of cooperative learning 
for developing the interpersonal skills required for effective team- 
work. 

In summary, there is broad empirical support for the central 
premise of cooperative learning, that cooperation is more effective 
than competition for promoting a range of positive learning out- 
comes. These results include enhanced academic achievement and a 
number of attitudinal outcomes. In addition, cooperative learning 
provides a natural environment in which to enhance interpersonal 
skills and there are rational arguments and evidence to show the ef- 
fectiveness of cooperation in this regard. 


D. Problem-Based Learning 

As mentioned in Section II of this paper, the first step of deter- 
mining whether an educational approach works is clarifying exactly 
what the approach is. Unfortunately, while there is agreement on 
the general definition of PBL, implementation varies widely. 
Woods et al. [16], for example, discuss several variations of PBL. 


“Once a problem has been posed, different instructional methods may be 
used to facilitate the subsequent learning process: lecturing, instructor- 
facilitated discussion, guided decision making, or cooperative learning. As 


part of the problem-solving process, student groups can be assigned to 
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complete any of the learning tasks listed above, either in or out of class. In the 
latter case, three approaches may be adopted to help the groups stay on track 
and to monitor their progress: (1) give the groups written feedback after each 
task; (2) assign a tutor or teaching assistant to each group, or (3) create fully 


autonomous, self-assessed “tutorless” groups.” 


The large variation in PBL practices makes the analysis of its ef- 
fectiveness more complex. Many studies comparing PBL to tradi- 
tional programs are simply not talking about the same thing. For 
meta-studies of PBL to show any significant effect compared to tra- 
ditional programs, the signal from the common elements of PBL 
would have to be greater than the noise produced by differences in 
the implementation of both PBL and the traditional curricula. 
Given the huge variation in PBL practices, not to mention differ- 
ences in traditional programs, readers should not be surprised if no 
consistent results emerge from meta-studies that group together 
different PBL methods. 

Despite this, there is at least one generally accepted finding that 
emerges from the literature, which is that PBL produces positive 
student attitudes. Vernon and Blake [19] looking at 35 studies from 
1970 to 1992 for medical programs found that PBL produced a sig- 
nificant effective size (0.55) for improved student attitudes and 
opinions about their programs. Albanese and Mitchell [20] similar- 
ly found that students and faculty generally prefer the PBL ap- 
proach. Norman and Schmidt [18] argue “PBL does provide a 
more challenging, motivating and enjoyable approach to education. 
That may be a sufficient raison d'etre, providing the cost of the im- 
plementation is not too great.” Note that these and most of the re- 
sults reported in this section come from studies of medical students, 
for whom PBL has been widely used. While PBL has been used in 
undergraduate engineering programs [49, 50] there is very little 
data available for its effectiveness with this population of students. 

Beyond producing positive student attitudes, the effects of PBL 
are less generally accepted, though other supporting data do exist. 
Vernon and Blake [19], for example, present evidence that there is a 
statistically significant improvement of PBL on students’ clinical 
performance with an effect size of 0.28. However, Colliver [22] 
points out that this is influenced strongly by one outlying study with 
a positive effect size of 2.11, which skews the data. There is also evi- 
dence that PBL improves the long-term retention of knowledge 
compared to traditional instruction [51-53]. Evidence also suggests 
that PBL promotes better study habits among students. As one 
might expect from an approach that requires more independence 
from students, PBL has frequently been shown to increase library 


use, textbook reading, class attendance and studying for meaning 
rather than simple recall [19, 20, 53, 54]. 

We have already discussed the problems with meta-studies that 
compare non-uniform and inconsistently defined educational inter- 
ventions. Such studies are easily prone to factors that obscure re- 
sults. The approach for handling this difficulty with active, collabo- 
rative and cooperative learning was to identify the central element 
of the approach and to focus on this rather than on implementation 
methods. That is more difficult to do with PBL since it is not clear 
that one or two core elements exist. PBL is active, engages students 
and is generally collaborative, all of which are supported by our pre- 
vious analysis. It is also inductive, generally self-directed, and often 
includes explicit training in necessary skills. Can one or two ele- 
ments be identified as common or decisive? 

Norman and Schmidt [18] provide one way around the difficul- 
ty by identifying several components of PBL in order to show how 
they impact learning outcomes. Their results are shown in Table 3, 
taken directly from Norman and Schmidt using the summary of 
meta-studies provided by Lipsey and Wilson [17]. The measured 
learning outcome for all educational studies cited by Lipsey and 
Wilson was academic achievement. 

Norman and Schmidt present this table to illustrate how differ- 
ent elements of PBL have different effects on learning outcomes. 
However, the substantive findings of Table 3 are also worth high- 
lighting for faculty interested in adopting PBL because there seems 
to be considerable agreement on what works and does not work in 
PBL. 

Looking first at the negative effects, there is a significant nega- 
tive effect size using PBL with non-expert tutors. This finding is 
consistent with some of the literature on helping students make the 
transition from novice to expert problem solvers. Research compar- 
ing experts to novices in a given field has demonstrated that becom- 
ing an expert is not just a matter of “good thinking” [42]. Instead, 
research has demonstrated the necessity for experts to have both a 
deep and broad foundation of factual knowledge in their fields. The 
same appears to be true for tutors in PBL. 

There is also a small negative effect associated with both self- 
paced and self-directed learning. This result is consistent with the 
findings of Albanese and Mitchell [20] on the effect of PBL on test 
results. In seven out of ten cases they found that students in PBL 
programs scored lower than students in traditional programs on 
tests of basic science. However, in three out of ten cases, PBL 
students actually scored higher. Albanese and Mitchell note that 
these three PBL programs were more “directive” than others, 


(e) Self-paced 
(f) Self-directed 


(g) Using problems 


Table 3. Effect sizes for academic achievement associated with various aspects of problem-based learning. 
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indicating that this element might be responsible for the superior 
exam performance for students in those programs. Therefore, facul- 
ty might be advised to be cautious about the amount of self-direc- 
tion required by students in PBL, at least with regard to promoting 
academic achievement as measured by traditional exams. 

Looking at what seems to work, there are significant positive ef- 
fect sizes associated with placing students in small groups and using 
cooperative learning structures. This is consistent with much of the 
literature cited previously in support of cooperative learning. While 
PBL and cooperative learning are distinct approaches, there is a 
natural synergy that instructors should consider exploiting. That is, 
real problems of the sort used in PBL require teams to solve effec- 
tively. At the same time, the challenge provided by realistic prob- 
lems can provide some of the mutual interdependence that is one of 
the five tenets of cooperative learning. 

Table 3 also shows that positive results come from instruction in 
problem solving. This is consistent with much of the advice given 
by proponents of problem-based learning [55]. While practice is 
crucial for mastering skills such as problem solving, greater gains are 
realized through explicit instruction of problem solving skills. How- 
ever, traditional engineering courses do not generally teach problem 
solving skills explicitly. Table 3 suggests that faculty using PBL 
consider doing just that. 

In conclusion, PBL is difficult to analyze because there is not 
one or two core elements that can be clearly identified with student 
learning outcomes. Perhaps the closest candidates for core elements 
would be inductive or discovery learning. These have been shown 
by meta-studies to have only weakly positive effects on student aca- 
demic achievement [56, 57] as measured by exams. This might ex- 
plain why PBL similarly shows no improvement on student test 
scores, the most common measure of academic achievement. 

However, while no evidence proves that PBL enhances academ- 
ic achievement as measured by exams, there is evidence to suggest 
that PBL “works” for achieving other important learning outcomes. 
Studies suggest that PBL develops more positive student attitudes, 
fosters a deeper approach to learning and helps students retain 
knowledge longer than traditional instruction. Further, just as co- 
operative learning provides a natural environment to promote inter- 
personal skills, PBL provides a natural environment for developing 
problem-solving and life-long learning skills. Indeed, some evi- 
dence shows that PBL develops enhanced problem-solving skills in 
medical students and that these skills can be improved further by 
coupling PBL with explicit instruction in problem solving. Similar- 
ly, supporting arguments can be made about PBL and the impor- 
tant ABET engineering outcome of life-long learning. Since self- 
directed learning and meta-cognition are common to both PBL 
and life-long learning, a logical connection exists between these de- 
sired learning outcomes and PBL instruction, something often not 
true when trying to promote life-long learning through traditional 
teaching methods. 


IV. CONCLUSIONS 


Although the results vary in strength, this study has found sup- 
port for all forms of active learning examined. Some of the findings, 
such as the benefits of student engagement, are unlikely to be con- 
troversial although the magnitude of improvements resulting from 
active-engagement methods may come as a surprise. Other findings 
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challenge traditional assumptions about engineering education and 
these are most worth highlighting. 

For example, students will remember more content if brief activi- 
ties are introduced to the lecture. Contrast this to the prevalent con- 
tent tyranny that encourages faculty to push through as much materi- 
al as possible in a given session. Similarly, the support for collaborative 
and cooperative learning calls into question the traditional assump- 
tions that individual work and competition best promote achieve- 
ment. The best available evidence suggests that faculty should struc- 
ture their courses to promote collaborative and cooperative 
environments. The entire course need not be team-based, as seen by 
the evidence in Springer et al. [43], nor must individual responsibility 
be absent, as seen by the emphasis on individual accountability in co- 
operative learning. Nevertheless, extensive and credible evidence sug- 
gests that faculty consider a nontraditional model for promoting aca- 
demic achievement and positive student attitudes. 

Problem-based learning presents the most difficult method to 
analyze because it includes a variety of practices and lacks a domi- 
nant core element to facilitate analysis. Rather, different implemen- 
tations of PBL emphasize different elements, some more effective 
for promoting academic achievement than others. Based on the lit- 
erature, faculty adopting PBL are unlikely to see improvements in 
student test scores, but are likely to positively influence student atti- 
tudes and study habits. Studies also suggest that students will retain 
information longer and perhaps develop enhanced critical thinking 
and problem-solving skills, especially if PBL is coupled with explicit 
instruction in these skills. 

Teaching cannot be reduced to formulaic methods and active 
learning is not the cure for all educational problems. However, there 
is broad support for the elements of active learning most commonly 
discussed in the educational literature and analyzed here. Some of 
the findings are surprising and deserve special attention. Engineer- 
ing faculty should be aware of these different instructional methods 
and make an effort to have their teaching informed by the literature 
on “what worl 
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